
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Ofltce 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



I ATTORNEY DOCKET NO. I CONFIRMATION NO. | 



APPLICATION NO. 



FILING DATE 



HRST NAMED INVENTOR 



09/751,147 



12/28/2000 



Todd Schocpflin 



21450 7590 05/20/2004 

STEVEN P KODA, KODA LAW OFFICE 
8070 E MILLPLAIN BLVD, No. 141 
VANCOUVER, WA 98664 



OT2.P68 



8715 



EXAMINER 



SIANGCHIN, KEVIN 



I ART UNIT I PAPER number" 



2623 

DATE MAILED: 05/20/2004 



1 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summary 



Application No. 

09/751,147 


Applicant(s) 

SCHOEPFLIN ET AL. 


cxaminor 

Kevin Siangchin 


Art Unit 

2623 





The MAILING DATE of this communication appears on the cover sheet ^{^^e correspondence address - 
Period for Reply ^ 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE jfK/IONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S. C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent temi adjustment. See 37 CFR 1.704(b). 

Status 

1 )D Responsive to communication(s) filed on . 

2a)S This action is FINAL. 2b)D This action is non-final. 

3) n Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) n Claim(s) 1-7.9-15 and 18-26 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim{s) is/are allowed. 

6) ^ Claim(s) 1-7,9-15 and 18-25 is/are rejected. 

7) K Claim(s) 26 Is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) 0 The drawing(s) filed on is/are: a)n accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 1 19 

12) n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)n All b)n Some * c)^ None of: 

1 .D Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.n Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

1 ) ^ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) 

2) □ Notice of Draflsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5) □ Notice of Infomnal Patent Application (PTO-152) 

Paper No(s)/Mall Date . 6) □ Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 



Office Action Summary 



Part of Paper No./Mail Date 8 



Application/Control Number: 09/751 ,147 Page 2 

Art Unit: 2623 

Detailed Action 



Drawings 

Response to Amendments to the Drawings 

1 . The corrected and/or new drawings were received on March 8, 2004. These drawings are approved. 



Specification 

Response to Amendments to the Specification 

2. The amendments to the specification filed on March 8, 2004 have been acknowledged. These amendments 
adequately address the deficiencies of the original specification. Therefore, the previous objections to the 
specification are withdrawn. 



Claims 

Response to Amendments to the Claims 

3. The amendment to the claims filed March 8, 2004 has been entered into the record and claims 1, 13, and 
18-19 have been amended accordingly. Claims 8 and 16-17 have been cancelled. Claims 24-26 have been added. 

Response to Arjguments and Remarks 

4. The following is in regard to the Applicant's assertion that Claim 1, as amended, distinguishes over prior 
art. Applicant's arguments filed on March 8, 2004 have been fully considered but are not persuasive for at least the 
following reasons. Note that reference will be made to the previous Office Action, particularly to the sections 
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relating to original Claim 1 and cancelled Claim 8. Please refer to paragraphs 1-7 and 23-25 of the previous Office 
Action. 

5. The Applicant amended claim 1 in such a way as to incorporate the limitations of cancelled claim 8 into the 
original claim 1. Due to the dependence of cancelled claim 8 on original claim 1, arguments set forth in the previous 
Office Action in regard to the 35 U.S.C. § 103(a) rejections of claims land 8 are, collectively, applicable to the 
amended claim 1. In particular, it was adequately shown in the previous Office Action that the combined teachings 
of Dickie (U.S. Patent 5,999,651), Geiger ("Dynamic Programming for Detecting, Tracking, and Matching 
Deformable Contours") and Sethian ("Level Set Methods and Fast Marching Methods") address the following 
aspects of amended claim 1 : 

(1 .a.) Receiving a set of control points used in deriving an object boundary from a prior image 

frame, wherein the received set of control points serves as an initial set of control points 

for the current digital image frame. 
(1 .b.) Defining a restricted area within the current digital image frame based on the received set 

of control points, wherein the restricted area corresponds to a band about the object 

boundary. 

(1 x.) Updating the initial set of control points to an updated set of control points in response to 
an operator command. 

(1 .d.) Deriving a cvurent-frame object boundary as a closed contour connecting each control 
point of the updated set of control points, wherein only image data within the restricted 
area are eligible to form the closed contour, wherein said derived closed contour serves 
as a current frame object boimdary. 
Furthermore, it was shown in the previous Office Action that the teachings of Sun et al. (U.S. Patent 6,546,1 17) can 
be combined with those of Dickie, Geiger and Sethian to satisfy the limitations of cancelled claim 8, which 
correspond to the following aspects of amended claim 1 : 
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Deriving a current-frame object boundary as a closed contour connecting each control point of the updated 
set of control points, such that, for each one control point of the updated set of control points, a path is 
derived from said one control point to an adjacent control point by: 
(1 .e.) Determining a distance from said one control point to said adjacent control point, 
(l.f.) When said distance is less than a threshold distance applying a first set of rules for 

deriving the path from said one control point to the adjacent control point, 
(l.g.) When said distance is greater than the threshold distance applying a second set of rules, 
different from the first set of rules, for deriving the path from said one control point to 
the adjacent control point, wherein said derived closed contour serves as a current frame 
object boundary. 

For the sake of brevity, the details of the arguments that were presented in the previous Office Action relating to 
these aspects of amended claim 1 are omitted. Please refer to paragraphs 1-7 and 23-25 of the previous Office 
Action. 

The Teachings of Sun et al (U.S. Patent 6,546,117). 

6. The Applicant correctly summarizes the pertinent teachings of Sun et al. in paragraph Ion page 15 of the 
amendment. 

7. The AppHcant argues that "the [Examiner's] characterization of Sun et al. does not involve the same steps 
as required by amended claim 1, nor does it suggest such steps". It will be assumed here that the Applicant is 
specifically referring to steps (l.e)-(l.g) above, since the teachings of Sun et al. were applied to similar limitations in 
the previous Office Action. Specifically, the Applicant asserts that according to method of Sun et al., "once the 
control points are obtained, however, there is no additional step of using a first set of rules or a second set of rules to 
derive a path to connect the two control points for which a distance was determined". On the contrary, the method of 
Sun et al. executes a first set of rules (i.e. eliminating the adjacent control points from the set of control points) 
when the measured distance between these control points is less than a first threshold, and executes a second set of 
rules (i.e. adding additional control points by interpolating the two adjacent control points) when the distance is 
greater than a second threshold (that threshold being greater than the first threshold). The word rule is interpreted 
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here as meaning a predetermined sequence of steps for the purposes of obtaining a certain result. In this manner, 
eliminating the adjacent control points from the set of control points, indeed, represents a rule - e.g. a first rule. 
Similarly, adding additional control points by interpolating the two adjacent control points represents a rule - e.g. a 
second rule. 

8. The applicant further asserts that, in the method of Sun et al., "the two control points used for determining 
distance are different than the control points for which a path is derived". While this is true in some cases, it does 
not mean Sun et al. fail to teach the aforementioned steps (l.e)-(l.g). In light of the preceding discussion, one 
realizes that this assertion is true only when the distance between the adjacent control points is less than the first 
threshold. However, the assertion is not valid for instances where the distance between the adjacent control points is 
greater than the second threshold. 

9. To illustrate this consider a subset of the set of control points, say, S = {Pi-\,Pi,Pi+i,Pi+2} > where Pi and 
are the adjacent control points under consideration. Assume the first threshold is T/ and second threshold is r?. If 

the distance between these adjacent control points is dij+j < T/ then, according the method of Sun et al, the 
points Pi and are eliminated. As a result, /?/.; and become adjacent and consequently the control points for 
which the path is derived. In this case, the control points (i.e. /?, and /?/+/) used for determining distance are different 
than the control points (i.e. and Pi+2) for which a path is derived. If, on the other hand, the distance between these 
adjacent control points is dij+j> r^then, according the method of Sun et al., the points pi and pi+j are interpolated, 
thereby generating, say, control points qj, q2„ ... ,qM between /?, and /?/+/. The set S then becomes 
^ = {Pi-\ J P/ 5 ^1 ) ^2 ^A/ ' > Pi+2 } • An optimal contour is subsequently derived that spans pi and (step 202, 
Sun et al. Fig. 1 1). Therefore, in instances where rf/ ,+/ > r^jthe control points (i.e. pi and /?,+;) used for determining 
distance are the same as the control points (i.e. pi and pi+j) for which a path is derived. 

10. Despite the validity, under certain circumstances, of the AppHcant's assertion that, in the method of Sun et 
al., "the two control points used for determining distance are different than the control points for which a path is 
derived", Sun et al. still teach the steps (l.e)-(l.g) above. First, when given its broadest interpretation a "set of rules 
for deriving a path from said one control point to the adjacent control point" can, indeed, include rules that result in 
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not deriving a path from said one control point to the adjacent control point. As a simplified analogy, consider rules 
for building a house. Rules for building a house may include mles for not building that house in certain 
circumstances (e.g. when building a house, do not build a house on unstable ground or when building a house, do 
not build the house if it costs too much). Seen in this light, the rule for eliminating the adjacent control points from 
the set of control points when the distance spanning the adjacent control points is less than the first predetermined 
threshold distance can still be considered a rule for deriving a path between the said one control point to the adjacent 
control point, despite the fact that, in this case, a path is not derived between the control points. In a manner similar 
to the analogy above, the first set of rule does not derive a path between the said one control point and the adjacent 
control point because it has been determined that circumstances (i.e. the distance between these points is less than 
the first threshold) do not warrant having a path between these points. In this way, the arguments presented in 
previous Office Action regarding canceled claim 8 adequately address the corresponding subject matter of claim 1 
and the arguments raised by the Applicant relating to the deficiencies of Sun et al.'s teachings. 

11. Defining the first set of rules for deriving the first set of rules in this manner is, admittedly, rather circuitous 
.The concems raised by the Applicant in regard to this definition are understandable. However, by redefining Sun et 
al.'s first set of rules for deriving a path between the said one control point and the adjacent control point, it can be 
shown, more explicitly, that Sun et al. teaches the steps (l.e)-(l.g) in such a way that the two control points used for 
determining distance are the same as the control points for which a path is derived. This is explained below. 

12, As mentioned previously, in the method of Sun et al., the distance between adjacent control points is 
compared to a second threshold (hereinafter, r?. The aforementioned first threshold will be denoted r/). In the case 
that the distance is greater than the second threshold, the method of Sun et al. derives a path between the said 
adjacent control points using a second set of rules. If, on the other hand, the distance is less that the second threshold 
(r?), the method of Sun et al. derives a path between the said adjacent control points using a first set of rules. The 
second set of rules is the same as that which was described above and in the previous Office Action. The first set of 
rules is redefined here to be the set of actions performed in the method of Sun et al. when the distance is less than 
the second threshold (r?) - that is, derive the optimal contour as usual (i.e. steps 200-202 in Sun et al. Fig. 1 1), for 
distance d such that <5< r?, or eliminate points when d < tj. See Sun et al. Fig. 1 1 and column 17, lines 65-67 to 
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column 18, lines 1-12. Therefore, Sun et al. teach: (l.e) Determining a distance, d, from said one control point to 
said adjacent control point, (l.f) when said distance is less than a threshold distance, r?, applying a first set of rules 
for deriving the path from said one control point to the adjacent control point (i.e. derive the optimal contour as 
usual [i.e. steps 200-202 in Sun et al. Fig. 1 1], for distance d such that Tj <S< r?, or eliminate points when d < t/. 
[Sun et al. Fig. 1 1 and column 17, lines 65-67 to column 18, lines 1-12]), and (l.g) when said distance is greater 
than the threshold distance applying a second set of rules, different from the first set of rules, for deriving the path 
from said one control point to the adjacent control point (i.e. adding additional control points by interpolating the 
two adjacent control points and deriving the contour, the original second set of rules), wherein said derived closed 
contour serves as a current frame object boundary. These observations and the discussion in the previous Office 
Action relating to original claim 1 serve as the basis for the 35 U.S. C. § 103(a) rejection of amended claim 1 below. 



Rejections Under 35 U.S.C. § 103(a) 

13. Reference will be made to the previous Office Action, which is incorporated herein in its entirety. See also 
the discussion above in response to the Applicant's arguments, 

14. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set 
forth in this Office action. 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if 
the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would 
have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

15. Claims 1-7, 9-10, 12-15, 18-19, and 22-25 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Dickie (U.S. Patent 5,999,651), in view Geiger et al. ("Dynamic Programming for Detecting, Tracking, and 
Matching Deformable Contours"), Sethian ("Level Set Methods and Fast Marching Methods"), and Sun et al. (U.S. 
Patent 6,546,1 17). 
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16. The following is in regard to Claim I. As shown above and in the previous Office Action, Dickie (U.S. 
Patent 5,999,651), Geiger ("Dynamic Programming for Detecting, Tracking, and Matching Deformable Contours") 
and Sethian ("Level Set Methods and Fast Marching Methods") can be combined so as to address the following 
aspects of amended claim 1: 

A method for segmenting an object within a current digital image frame, comprising the steps of: 

( 1 .a.) Receiving a set of control points used in deriving an object boundary from a prior image 

frame, wherein the received set of control points serves as an initial set of control points 

for the current digital image frame, 
(l.b.) Defining a restricted area within the current digital image frame based on the received set 

of control points, wherein the restricted area corresponds to a band about the object 

boundary. 

(I.e.) Updating the initial set of control points to an updated set of control points in response to 
an operator command. 

(l.d.) Deriving a current-frame object boundary as a closed contour connecting each control 
point of the updated set of control points, wherein only image data within the restricted 
area are eligible to form the closed contour, wherein said derived closed contour serves 
as a current frame object boundary, 

17. For the sake of brevity, the details presented in the previous Office Action relating to steps (1 .a)-(l.d) will 
be omitted here. Please refer to the appropriate sections of the previous Office Action. 

18. Dickie, Geiger, and Sethian do not, however, show the following steps: 

Deriving a current-frame object boundary as a closed contour connecting each control point of the updated 
set of control points, such that, for each one control point of the updated set of control points, a path is 
derived from said one control point to an adjacent control point by: 
(I.e.) Determining a distance from said one control point to said adjacent control point, 
(l.f ) When said distance is less than a threshold distance applying a first set of rules for 
deriving the path from said one control point to the adjacent control point. 
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(l.g.) When said distance is greater than the threshold distance applying a second set of rules, 
different from the first set of rules, for deriving the path from said one control point to 
the adjacent control point, wherein said derived closed contour serves as a current frame 
object boundary. 

19. As discussed extensively above, Sun et al, teach the following: 

Deriving a current-frame object boundary as a closed contour connecting each control point of the updated 

set of control points, such that, for each one control point of the updated set of control points, a path is 

derived from said one control point to an adjacent control point by: 

(1 .e.) Determining a distance from said one control point to said adjacent control point. 

(1 .f ) When said distance is less than a threshold distance applying a first set of rules for 

deriving the path from said one control point to the adjacent control point, 
(l.g.) When said distance is greater than the threshold distance applying a second set of rules, 

different from the first set of rules, for deriving the path from said one control point to 

the adjacent control point, wherein said derived closed contour serves as a current frame 

object boundary. 
For the sake of brevity, the details will not be repeated here. 

20. It was adequately shown in the previous Office Action that teachings Dickie, Geiger, Sethian, and Sun et al. 
were combinable. The motivation for combining Dickie, Geiger, and Sethian were provided in the previous Office 
Action and remains applicable to those aspects of amended claim 1 common to original claim 1. See the appropriate 
sections of the previous Office Action, The motivation for incorporating the teachings of Sun et al. into the 
combined teachings of Dickie, Geiger, and Sethian is similar to that which was given in the previous Office Action. 
The advantage of incorporating steps (l.e)-(l.g) into the segmentation method, obtained by combining the teachings 
of Dickie, Geiger, and Sethian, in the manner described above and in the previous Office Action, is that it provides 
an "adaptive" refinement of the contour. When control points are too distant {d > T2), points are added (i.e. the 
contour is refined), thereby making the contour more responsive to the local features of the object boundary. Taking 
this into account, it would have been obvious to one of ordinary skill in the art, at the time of the applicant's claimed 





Application/Control Number: 09/751,147 
Art Unit: 2623 



Page 10 



invention, to incorporate the teachings of Sun et al., as described above and in the previous Office Action, into the 
segmentation method, obtained by combination of Dickie, Geiger, and Sethian, The segmentation method thus 
obtained conforms to the method set forth in claim 1 . 

21. The following is in regard to Claim 2. As shown above, the teachings of Dickie, Geiger, Sethian, and Sun 
et al. can be combined so as to satisfy the limitations of claim 1. As shown in the previous Office Action, Dickie 
teaches the step of updating comprises updating the initial set of control points to an updated set of control points in 
response to an operator command to perform either one of moving a control point or adding a control point. For the 
sake of brevity, the details presented in the previous Office Action relating to this step will be omitted here. Please 
refer to the section of the previous Office Action relating to original claim 2. Therefore, the teachings of Dickie, 
Geiger, Sethian, and Sun et al. can be combined to yield a segmentation method in accordance with claim 2 

22. The following is in regard to Claim 3. As shown above, the teachings of Dickie, Geiger, Sethian, and Sun 
et al. can be combined in such a way as to satisfy the limitations of claim 2. As shown in the previous Office Action, 
Sethian teaches the step of redefining the restricted area to encompass the select control point, when the operator 
command either one of moves or adds a select control point outside the restricted area. For the sake of brevity, the 
details presented in the previous Office Action relating to this step will be omitted here. Please refer to the section of 
the previous Office Action relating to original claim 3. The teachings of Dickie, Geiger, Sethian, and Sun et al. can 
thus be combined to yield a segmentation method in accordance with claim 3. 

23. The following is in regard to Claim 4. As shown above, the teachings of Dickie, Geiger, Sethian, and Sun 
et al. can be combined in such a way as to satisfy the limitations of claim 1. As shown in the previous Office Action, 
Dickie teaches the steps of: 

(4.a.) Deriving edge energy data for the current image fi-ame. 

(4.b.) Deriving and displaying an initial estimate of the current-frame object boundary based on 



For the sake of brevity, the details presented in the previous Office Action relating to these steps will be omitted 
here. See sections of the previous Office Action related to original claim 4. Note these steps are illustrated in Dickie 
Fig. 5. From Dickie Fig. 3, it is clear that these steps (encompassed in Dickie Fig. 3 step 40) precede the step of 



the received set of control points and the derived edge energy data. 
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updating (e.g. steps 42-44 in Dickie Fig, 3). Thus, Dickie addresses the limitations of claim 4. Therefore, the 
teachings of Dickie, Geiger, Sethian, and Sun et al. can be combined to yield a segmentation method in accordance 
with claim 4. 

24. The following is in regard to Claim 5. As shown above, the teachings of Dickie, Geiger, Sethian, and Sun 
et al. can be combined so as to satisfy the limitations of claim 4. As shown in the previous Office Action, Dickie 
teaches that the current digital image frame is displayed and the initial estimate of the current-frame object boundary 
is overlaid onto the current digital image frame with the initial set of control points frame, wherein the step of 
updating the initial set of control points con:q)rises selecting a control point of the initial set of control points with a 
pointing device and dragging the selected control point to a new location. For the sake of brevity, the details 
presented in the previous Office Action relating to this matter will be omitted here. See the sections of the previous 
Office Action related to original claim 5. The teachings of Dickie, Geiger, Sethian, and Sun et al. can thus be 
combined to yield a segmentation method in accordance with claim 5. 

25. The following is in regard to Claim 6. . As shown above, the teachings of Dickie, Geiger, Sethian, and Sun 
et al. can be combined in such a way as to satisfy the limitations of claim 5. As shovm in the previous Office Action, 
the steps of : 

(6.a.) Testing the new location of the selected control point to determine whether the selected 

control point is outside the restricted area. 
(6.b.) When the new location of the selected control point is outside the restricted area, 

redefining the restricted area to encompass the selected control point in the new location, 
follow from Sethian's teachings relating to the re-initialization of the Narrow Band Level Set Method. For the sake 
of brevity, the details presented in the previous Office Action relating to these steps will be omitted here. See the 
sections of the previous Office Action related to original claim 6. 

26. The following is in regard to Claim 7. As shown above, the teachings of Dickie, Geiger, Sethian, and Sun 
et al. can be combined so as to satisfy the limitations of claim 5. As shown in the previous Office Action, Dickie 
teaches that the step of defining the current-frame object boundary comprises deriving a path from the selected 
control point to a first adjacent control point and deriving a path from the selected control point to a second adjacent 
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control point. . For the sake of brevity, the details presented in the previous Office Action relating to this step will be 
omitted here. See the sections of the previous Office Action related to original claim 7. Therefore, the teachings of 
Dickie, Geiger, Sethian, and Sun et al. can be combined to yield a segmentation method in accordance with claim 7. 

27. The following is in regard to Claim 9. As shown above, the teachings of Dickie, Geiger, Sethian, and Sun 
et al. can be combined in such a way as to satisfy the limitations of claim 1 . Note that, despite the change m its claim 
dependency, amended claim 9 proposes essentially the same subject matter as claimed in original claim 9. It was 
shown in the previous office action that the second set of mles for connecting control points, when evaluated within 
the context of the segmentation method derived from the combined teachings of Dickie, Geiger, Sethian, and Sun et 
al., is less accurate than first set of rules (in terms of the derived pafli), in the cases when the distance between the 
adjacent control points is sufficiently small (e.g. when d < rl). The analysis presented there is still applicable to the 
present discussion, despite the difference in the definition of the furst set of rules. Therefore, the teachings of Dickie, 
Geiger, Sethian, and Sun et al. can be combined to yield a segmentation method in accordance with claim 9. 

28. The following is in regard to Claim JO, As shown above, the teachings of Dickie, Geiger, Sediian, and Sun 
et al. can be combined in such a way as to satisfy the limitations of claim 1. Note that, despite the change in its claim 
dependency, amended claim 10 proposes essentially the same subject matter as claimed in original claim 10. It was 
shown in the previous office action that the second set of rules for connecting control points, when evaluated within 
the context of the segmentation method derived from the combiried teachings of Dickie, Geiger, Sethian, and Sun et 
al, is more accurate than fu-st set of rules (in terms of the derived path), in the cases when the distance between the 
adjacent control points is sufficiently large (e.g. when d > T2). The analysis presented there is still applicable to the 
present discussion, despite the difference in the definition of the fu-st set of rules. Therefore, the teachings of Dickie, 
Geiger, Sethian, and Sun et al. can be combined to yield a segmentation method in accordance with claim 10. 

29. The following is in regard to Claim 12. As shown above, the teachings of Dickie, Geiger, Sethian, and Sun 
et al. can be combined in such a way as to satisfy the limitations of claim 1. As shown in the previous Office Action, 
Geiger et al. teaches the step of receiving an object boundary estimate of a prior digital image frame and wherein the 
step of defining the restricted area, comprises defming the restricted area within the current digital image frame from 
the received object boundary estimate, including the received set of control points. For the sake of brevity, the 
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details presented in the previous Office Action relating to this step will be omitted here. See sections of the previous 
Office Action related to original claim 12. The teachings of Dickie, Geiger, Sethian, and Sun et al. can dius be 
combined to yield a segmentation method in accordance with claim 12. 

30. The following is in regard to Claims 13-15 and 18-19 . Note that the apparatuses set forth in claim 13, 14, 
15, and 18-19 are essentially physical manifestations of the segmentation methods proposed in claims 1,3,6, and 9- 
10, respectively. Therefore, with regard to claims 13, 14, 15, and 9-10, arguments analogous to those presented 
above relating to the rejections of claim 1, 3, 6, and 9-10 are, respectively, applicable. See the discussion above 
relating to claims 1, 3, 6, and 9-10. 

31. The following is in regard to Claim 22. As shown above, the teachings of Dickie, Geiger, Sethian, and Sun 
et al. can be combined to yield a segmentation apparatus in accordance with claim 13. As pointed out in the previous 
Office Action, it would have been obvious to one of ordinary skill in the art to perform all operations of the 
segmentation apparatus of claim 13 using a single processor. Clearly, by doing this one reduces the hardware 
complexity of the segmentation apparatus. In this way, the segmentation apparatus, obtained by the combination of 
Dickie, Gieger, Sethian, and Sun, can be trivially modified to satisfy the limitations of claim 22. 

32. The following is in regard to Claim 23. As shown above, the teachings of Dickie, Geiger, Sethian, and Sun 
et al. can be combined to yield a segmentation apparatus in accordance v^dth claim 13. Note that the apparatus of 
claim 23 is essentially a physical manifestation of the method of claim 5. Therefore, with regard to claim 23, 
arguments analogous to those presented above relating to claim 5 are applicable. See the discussion above relating to 
the rejections of claims 4 and 5. 

33. The following is in regard to Claim 24. As shown above, the teachings of Dickie, Geiger, Sethian, and Sun 
et al. can be combined in such a way as to satisfy the limitations of claim 1. Sethian's Fast Marching Method 
(Sethian pp. 86-100) has been widely used in image segmentation applications. See, for exan^le, Sethian Section 
17.2.3 on pages 221-226. Segmentation is achieved by propagating a contour (an evolving front) - i.e. deriving a 
path - to conform to the boundary of an object of interest. See, for example. Fig. 17.7 on page 224 of Sethian. The 
Fast Marching Method (FMM) attempts to solve the front propagation problem by casting it in terms of solving the 
Eikonal equation: 
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(24.1) 



See Sethian page 5. Since ||vr| > 0, (24. 1) implies that the speed F > 0, that is, the front propagates outward or 

forward always. See the discussion on page 4 (last paragraph) to page 5 of Sethian. Furthermore, when propagating 
the front, the FMM considers grid points (i.e. pixels, in the case of image data) within the boundary of a "narrow 
band of trial values " (Sethian Section 8,3 on page 90 and Fig. 8.4). In this way, the FMM can be regarded as a 
forward or outward marching approach to deriving the path between control points, wherein the path is constrained 
to a restricted area, namely the narrow band of trial values. Note that this narrow band is analogous to the narrow 
band of the Sethian' s Narrow Band Level Set Method (see the discussion in the previous Office Action regarding 
original claim 1). The restricted area, as defined here, can thus be considered the same as the restricted area of claim 
1 . It would have been obvious to one of ordinary skill in the art, at the time of the applicant's claimed invention, to 
use the FMM as the first set of mles for deriving a path in the segmentation method, obtained by the aforementioned 
combination of Dickie, Geiger et al., Sethian, and Sun et al.'s teachings, as opposed to the first set of rules taught by 
Sun et al. The motivation to use the FMM in lieu of Sun et al.'s first set of rules would have been the fact that the 
FMM, as in the Level Set Method, naturally accommodates high-curvature regions and/or topological changes in the 
boundary of the object of interest (Sethian page 8, second bullet)*. Integrating, in this manner, the FMM into the 
segmentation method, obtained by combining the teachings of Dickie, Geiger et al, Sethian, and Sun et al. as 
discussed above, yields a method that conforms to limitations set forth in new claim 24. 

34. The following is in regard to Claim 25, As shown above, the teachings of Dickie, Geiger, Sethian, and Sun 
et al. can be combined in such a way as to satisfy the limitafions of claim 1. Sethian's Fast Marching Method 
(Sethian pp. 86-100) has been widely used in image segmentation. See, for example, Sethian Section 17.2.3 on 
pages 221-226. Segmentation is achieved by propagating a contour (an evolving front) - i.e. deriving a path - to 
conform to the boundary of an object of interest. See, for example, Fig. 17.7 on page 224 of Sethian. The Fast 



Active contour models, such as those utilized in the method of Sun et al. (Sun et al. Fig.2, step 52), are known to become unstable or fail ii 
regions of high curvature or where the topology changes. 
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Marching Method (FMM) attempts to solve the front propagation problem by casting it in terms of solving the 
Eikonal equation: 

||Vr||F = l (25.1) 

35. See Sethian page 5. Since ||vr| > 0, (24.1) implies that the speed F>0, that is, the front propagates 

outward or forward always. See the discussion on page 4 (last paragraph) to page 5 of Sethian. Furthermore, when 
propagating the front, the FMM considers grid points (i.e. pixels, in the case of image data) within the boundary of a 
"narrow band of trial values" (Sethian Section 8.3 on page 90 and Fig. 8.4). In this way, the FMM can be regarded 
as a forward or outward marching approach to deriving the path between control points, wherein the path is 
constrained to a restricted area, namely the narrow band of trial values. Note that this narrow band is analogous to 
the narrow band of the Sethian's Narrow Band Level Set Method (see the discussion in the previous Office Action 
regarding original claim 1). The restricted area, as defined here, can thus be considered the same as the restricted 
area of claim 1. It would have been obvious to one of ordinary skill in the art, at the time of the applicant's claimed 
invention, to use the FMM as the second set of rules for deriving a path in the segmentation method, obtained by the 
aforementioned combination of Dickie, Geiger et al., Sethian, and Sun et al.'s teachings, as opposed to the second 
set of rules taught by Sun et al. The motivation to use the FMM in lieu of Sun et al.'s second set of rules would have 
been the fact that the FMM, as in the Level Set Method, naturally accommodates high-curvature regions and/or 
topological changes in the boundary of the object of interest (Sethian page 8, second bullet)^. Integrating, in this 
manner, the FMM into the segmentation method, obtained by combining the teachings of Dickie, Geiger et al., 
Sethian, and Sun et al. as discussed above, yields a method that conforms to limitations set forth in new claim 25. 

36. Claims 1 1 and 20-21 are rejected under 35 U.S.C. 103(a) as being unpatentable over Dickie, in view Geiger 
et al., Sethian, and Sun et al., in further view of Guo et al. ("New Video Object Segmentation Technique with 
Color/motion Information and Boundary Postprocessing"). 



Active contour models, such as those utilized in the method of Sun et al. (Sun et al. Fig.2, step 52), are known to become unstable or fail ii 
regions of high curvature or where the topology changes. 
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37. The following is in regard to Claim II. As shown above, the teachings of Dickie, Geiger et al., Sethian, and 
Sun et al. can be combined in such a way as to satisfy the limitations of claim 1. However, neither of Dickie, Geiger 
et al., Sethian, and Sun et al teach that the step of defining the restricted area comprises morphologically dilating a 
contour connecting the set of control points. 

38. As shown in the previous Office Action, Guo et al. teach morphologically dilating the contour connecting 
the control points in order to obtain a restricted area about that contour. For the sake of brevity the details relating of 
this process and its compatibility with the methods of Dickie, Geiger et al, and Sethian are omitted here. See the 
discussion relating to the rejection of claim 1 1 in the previous Office Action. Since morphological dilation is 
performed on the derived contour, the addition of the aforementioned teachings of Sun et al., does not affect this 
compatibility. Therefore, it would have been obvious to one of ordinary skill in the art, at the time of the applicant's 
claimed invention, to morphologically dilate the derived contour in order to obtain a restricted area about the 
contour. Again, one is motivated to use morphological dilation of the derived contour, in the manner taught by Guo 
et al., in order to derive a restricted area about the contour, because the morphological dilation operation is a simple, 
efficient, and often-used region growing technique. The segmentation method thus obtained would conform to the 
limitations set forth in claim 11. 

39. The following is in regard to Claim 20. Note that claim 20 recites substantially the same limitations as 
claim 1 1. Therefore, with regard to claim 20, arguments analogous to those presented above with respect to claim 1 1 
are applicable, 

40. The following is in regard to Claim 21. As just shown, the teachings of Dickie, Geiger et al., Sethian, Sun 
et al., and Guo et al. can be combined in such a way as to satisfy the limitations of claim 20. The following is an 
excerpt taken from Geiger, et al. (see Geiger, et al., the first to third paragraphs under Section III, "Tracking 
Deformable Contours" in the attached Errata): 

Suppose that we are given a sequence of images ... and a deformable contour associated with an object in the first frame. How can we 
use this information to automatically detect the subsequent contours? 

. . . [T]he new contour should be close to the previous one. . . First, we simply copy the contour to the next frame and sample at the 
extrema points ... We consider these points to be the input, much like the ones provided by the user in the initial frame. 
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It should be clear from this excerpt that the contour of a previous image frame (e.g. the first image frame) serves as 
the object boundary estimate from a prior image frame, in accordance with the applicant's claim 12. Therefore, in 
tracking the contour representing the boundary of the object of interest, an object boundary estimate from a prior 
frame is received. Further note that, according to the excerpt above, a set of control points from the prior image 
frame (i.e. the points sampled from the previous contour) are also received. Geiger, et al. also show, in the above 
excerpt, that the object boundary estimate from a prior frame, itself, provides an initial estimate for the contour that 
represents the boundary of the object of interest with respect to the current frame. This, of course, facilitates the 
derivation of the minimal-cost closed contour that represents the boundary of the object of interest with respect to 
the current frame. Again, one of ordinary skill in the art could construct a narrow band about the received object 
boundary estimate, in the manner of Sethian's Narrow Band Level Set Method (see paragraph 7 above), thus, 
defining a restricted area in accordance v^th claim 12.. The motivation to do so is the same as that which was 
presented above for claim 1. Thus, given the teachings of Geiger, et al, as they are present here and the teachings of 
Sethian, as applied to claim 1, it would have been it would have been obvious to one of ordinary skill in the art, at 
the time of the applicant's claimed invention, to construct an image segmentation method, in accordance with claim 
1 , that receives an object boundary estimate from a prior frame and defines a narrow band restricted area about that 
boundary estimate. Arguments presented above with regard to claim 1 further support this conclusion. Thus, the 
teachings of Dickie, Geiger et al., Sethian, Sun et al., and Guo et al., when combined as discussed above, yield an 
apparatus, in accordance with claim 20, wherein said contour is an object boundary estimate from a prior digital 
frame that includes the initial set of control points. 



Objections, Allowable Subject Matter 

41 . Claim 26 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten 
in independent form including all of the limitations of the base claim and any intervening claims. 



Allowable Subject Matter 
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42. The following is a statement of reasons for the indication of allowable subject matter. 

Note that the subject matter claimed in claim 26 is the combmation of the proposed methods of claim 24 and claim 
25. It was shown above that, given the prior art available at the time of the applicant's claimed invention, claims 24 
and 25 would have been obvious to one of ordinary skill in the art. In particular, it was shown above that the FMM 
could be reasonably considered as a forward or outward marching approach to deriving the path between control 
points, wherein the path is constrained to a restricted area, namely the narrow band of trial values. However, no prior 
art was encountered that suggested that the methods of claims 24 and 25 could be combined. Specifically, prior art 
was not found that showed independently, or could be reasonably combined to show, an apparatus, in accordance 
with claim 13, wherein the contour deriving means comprises a means for applying forward marching method 
approach to determine the path and a means for applying an outward marching approach to determine the path, such 
that the said path occurs entirely within the restricted area, and the said forward marching method applying means is 
active upon application of the furst set of rules for deriving a path, and the said outward marching method applying 
means is active upon application of the second set of rules for deriving a path. 



Conclusion 

43. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time pohcy as set forth in 
37CFR 1.136(a). 

44. A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the 
mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final 
action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, 
then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee 
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pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, 
will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. 
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